I certainly understand the benefit of moving the request_start into the base request. However, I don't see the point of having the prototype of this function with a count and an array of request, as in general we are not able to alter the order of requests in the user array. Do we really expect to find ranges of similar requests and start them together ?

@bosilca Certainly the prototype of ompi_request_start_fn_t looks a little odd. But the point is giving a chance to optimize communication scheduling under the rule of the MPI Standard. There is a study on this.
Masayuki Hatanaka, Atsuhi Hori, and Yutaka Ishikawa, "Optimization of MPI Persistent Communication," Submitted to EuroMPI 2013, 2013.

I have little doubt about the interest of having persistent requests. I was asking about the need to have the start function associated with a particular request having the count and array of requests arguments, instead of simply using a request.

Let's assume:

1. Multiple components (ob1 PML, my COLL, ...) provide persistent requests through MPI_*_INIT.
2. Each component has a logic to start its multiple requests and may optimize the communication scheduling of those request.
3. Requests passed to MPI_STARTALL may consist of requests from multiple components.

In this situation, MPI_STARTALL must identify the owner component of each request and call startall functions of the corresponding components.

For optimization of 2., a function having only one request argument is not suited.

My logic was:

1. In order to identify the owner component, the ompi_request_t structure should have a new member.
2. In order to call the corresponding startall function, using the function pointer as the new member is simple.

Of course, there are other ways to associate a request with a startall function.

enum persistent_module_type {
    PERSISTENT_MODULE_TYPE_NULL,
    PERSISTENT_MODULE_TYPE_PML,
    PERSISTENT_MODULE_TYPE_COLL,
   ...
}

union persistent_module_t {
    void                   *ptr;
    mca_pml_base_module_t  *pml;
    mca_coll_base_module_t *coll;
    ....
}

struct ompi_request_t {
    ....
    enum persistent_module_type module_type;
    union persistent_module_t   module;
}

int MPI_Startall(...)
{
    union persistent_module_t module = { .ptr = NULL };
    for (i = 0, j = -1; i < count; ++i) {
        // Call a callback function per block of requests
        // which belong to the same module
        if (requests[i]->module.ptr != module.ptr && i != 0) {
            switch (requests[i - 1]->module_type) {
            case PERSISTENT_MODULE_TYPE_PML:
                module.pml->start(i - j, requests + j);
                break;
            case PERSISTENT_MODULE_TYPE_COLL:
                module.coll->start(i - j, requests + j);
                break;
            ...
            }
            module.ptr = requests[i]->module.ptr;
            j = i;
        }
    }
    // the last requests block
    switch (requests[i - 1]->module_type) {
    case PERSISTENT_MODULE_TYPE_PML:
        module.pml->start(i - j, requests + j);
        break;
    case PERSISTENT_MODULE_TYPE_COLL:
        module.coll->start(i - j, requests + j);
        break;
    ...
    }
}

Is there more smart solution?

Right, this solution rely heavily on the assumption that the user will place the persistent requests in the array for startall based on their logical generator. That's the only point I was trying to make.

Please update the padding for the predefined_request_t in request.h to reflect the addition of the extra function pointer.

@bosilca Do we need to update the padding of ompi_predefined_request_t? ompi_predefined_request_t is 256 bytes regardless of ompi_request_t, which is at most 192 bytes or so.

Current definition in request.h:

#define PREDEFINED_REQUEST_PAD 256

struct ompi_predefined_request_t {
    struct ompi_request_t request;
    char padding[PREDEFINED_REQUEST_PAD - sizeof(ompi_request_t)];
};

I think we concluded during the face-to-face meeting that there is no need to change the padding.